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Abstract. Let A4 be a compact Riemannian manifold of dimension n. The fc-curvature, 
for k = 1, 2, • • • , n, is defined as the fc-th elementary symmetric polynomial of the eigenval- 
ues of the Schouten tenser. The fc- Yamabe problem is to prove the existence of a conformal 
metric whose fc-curvature is a constant. When fc = 1, it reduces to the well-known Yamabe 
problem. Under the assumption that the metric is admissible, the existence of solutions to 
the fc- Yamabe problem was recently proved by Gursky and Viaclovsky for fc > ^. In this 
paper we prove the existence of solutions for the remaining cases 2 < fc < ^ , assuming that 
the equation is variational. 



1. Introduction 

In recent years the Yamabe problem for the /c-curvature of the Schouten tensor, or 
simply the k- Yamabe problem, has been extensively studied. Let (Ai,go) be a compact 
Riemannian manifold of dimension n. Denote by Riem, Ric, and R the Riemannian 
curvature tensor, the Ricci tensor, and the scalar curvature, respectively. Then one has 
the standard decomposition Riem = W + A go, where W is the Weyl tensor, A is 
the Schouten tensor given in (1.2) below, and denotes the Kulkarni-Nomizu product 
[B]. As the Weyl curvature tensor is conformally invariant, the transformation of the 
Riemannian curvature tensor under conformal changes of metrics is determined by that 
of the Schouten tensor. Therefore it is of interest to study curvature functions of the 
Schouten tensor under conformal deformation. A fundamental problem is the fc- Yamabe 

4 

problem, to prove the existence of a conformal metric g = g v = t>™- 4 (?o whose /c-curvature 
is equal to a constant, that is 

a k (X(A g )) = l, (1.1) 

where 1 < k < n is an integer, A = (Ai, • • • , A n ) are the eigenvalues of A g with respect 
to the metric g, and 

A, = ^(«c 3 - ) (1.2) 
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is the Schouten tensor. As usual we denote by 

a k (X)= *ii (1-3) 

ii<---<ik 

the k-th elementary symmetric polynomial. When k = 1, we arrive at the well known 
Yamabe problem. 

When k > 2, the /c- Yamabe problem was initiated by Viaclovsky [VI] and also arose 
in the study of the Paneitz operator [Br, CGY1]. Equation (1.1) is a fully nonlinear 
partial differential equation. To work in the realm of elliptic operators, one assumes that 
the eigenvalues X(A g ) lie in the convex cone [CNS], where 

T k = {A e R n | <jj (A) > for j = 1, • • • , k}. (1.4) 

Under this assumption, the k- Yamabe problem has been solved in the cases when n = 4 
and k = 2 [CGY1, CGY2], or when the manifold is locally conformally flat [LL1, GW2]. 
Very recently Gursky and Viaclovsky [GV2] solved the problem for k > ^, using the 
positivity of the Ricci curvature in this case. 

In this paper we employ a variational method to treat the problem for the cases 
2 < k < Tj. We prove that equation (1.1) has a solution as long as it is variational, 
namely it is the Euler equation of a functional, which includes the cases when k = 2 
and when Ai is locally conformally flat. In Section 2 we give a sufficient and necessary 
condition for equation (1.1) to be variational. 

When (1.1) is variational and k ^ ^, its solutions correspond to critical points of the 
functional 

n — 2 f n — 2 f 

J{g) = W^k) J M ak{x{9))dvd * ~^m] m dvol ° (L5) 
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in the conformal class [go] = {g \ g = v^^go, v > 0}. When k = ^, the first integral 
in (1.5) is a constant and we need to replace it by (2.32) below. We will find a min-max 
(Mountain Pass Lemma) solution, as in the case k = 1. Note that in the case k > ^ , the 
coefficients are negative and the functional is negative. 

The progressive resolution of the Yamabe problem (k = 1) by the second author, 
Aubin and Schoen [Ya, Trl, Aul, SI] was a milestone in differential geometry. Roughly 
speaking, the overall proof consists of two parts. The first one is to show that the Yamabe 
problem is solvable if the Yamabe constant Y\ satisfies the condition 

Y 1 (M)<Y 1 (S n ), (1.6) 

and the second one is to verify the condition (1.6) for manifolds not conformally diffeo- 

morphic to the unit sphere S n with standard metric. When M is locally conformally 

flat, different proofs were found later [SY1, Ye]. 
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For the /c-Yamabe problem, 2 < k < ^, our variational approach basically comprises 
the same two steps. Namely one first shows that (1.1) has a solution when the /c-Yamabe 
constant Y k satisfies 

Y k (M) < Y k (S n ), (1.7) 

and then verify the condition (1.7) for manifolds not conformal to the unit sphere S n . 
But since equation (1.1) is fully nonlinear, our treatment is technically different and more 
complicated. For the first step, we cannot apply the variational method directly, since 
we need to restrict the functional (1.5) to a subset of the conformal class [go], given by 

[9o]k = {g\ 9 = v^g , v>0, X(A g ) e T fc }, (1.8) 
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and the set of functions v with v n ~ 2 go G [go]k may not be convex. Through the functional 
(1.5), we introduce a descent gradient flow, establish appropriate a priori estimates, and 
prove the convergence of solutions to the flow under assumption (1.7). We need to choose 
a particular gradient flow to obtain the a priori estimates, locally in time. 

For the second step, it seems impossible to find an explicit /c-admissible test function. 
The function 

which is the unique solution of (1.1) on the Euclidean space R n for all 1 < k < n, is k- 
admissible only when r < Ce 1 / 2 on a general manifold, where r is the geodesic distance. 
Fortunately we found a simple way to deduce (1.7) directly from (1.6). 

This paper is arranged as follows. In Section 2, we state the main results, specifically 
in §2.1, while in §2.2 we outline the proof. In §2.3 we collect some related results on the 
/c-Hessian equation. In §2.4 we give a sufficient and necessary condition for a partial dif- 
ferential equation to be variational. In Section 3 we study the regularity for the gradient 
flow of the functional (1.5) for solutions with \(A 9v ) e T k: and give counterexamples to 
interior regularity for solutions with X(A 9v ) e (— T k ). In Section 4 we investigate the 
asymptotic behavior of a descent gradient flow and prove the convergence of the flow 
under condition (1.7). We then prove (1.7) for manifolds not conformal to S n in Section 
5. The final Section 6 contains some remarks. 

The authors are grateful to Kaiseng Chou for useful discussions. This research was 
largely carried out in the winter of 2004-05 while the third author was at the Nankai 
Institute of Mathematics in China under a Yangtze River Fellowship. The other authors 
are also grateful for the Nankai Institute for hospitality when we were all there together 
in November 2004. 
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2. The main results 
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2.1. The main results. Let (M.,go) be a Riemannian manifold. If g = v go is a 
solution of (1.1), then the Schouten tensor is given by A g = j^z^V ', and u satisfies the 
equation 



L[v] :=v {1 - k ^a k (X(V)) = v^* , (2.1) 



where 



V = -V 2 v + 



n-2 



(2.2) 



n — 2 v 



n — 2 u 



0o + 
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Equation (2.1) is a fully nonlinear equation of similar type to the /c-Hessian equations 
[CNS, CW2, II, TW2]. For the operator L to be elliptic, we need to restrict to metrics 
with eigenvalues X(A g ) G U(±1V), which we will simply denote as g G ±IV Therefore 
equation (2.1) has two elliptic branches, one is when the eigenvalues A G Tk and the 
other one is when A G (— IV). In this paper we will mainly consider solutions with 
eigenvalues in IV Accordingly we say v is k- admissible (that is v is strictly subharmonic 
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with respect to L) if g = g G IV The set of all fc-admissible functions will be 
denoted by $fc = &k(M,go). In this paper we will always assume, unless otherwise 
indicated, that 2 < k < ^ and the following two conditions hold, 



(C2) The operator L is variational. 

The condition (CI) ensures that the operator L is elliptic, and may be replaced by 
Yj(M) > for j = 1, • • • , k, as in the case when k = 2 and n = 4 [CGY1, GV1]. Note 
that in condition (CI), we do not assume directly that the metric go G IV rather we 
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assume that there exists a positive function v such that v n ~ 2 go G IV Conditions (CI) 
(C2) are automatically satisfied when k = 1. 

As for the Yamabe problem, we introduce the fc-Yamabe constant for 2 < k < ^, 



(CI) The set <S> k (M, go) ^ 0; 



Y k (M) 



mf{F k (g) \ge[g } k ,Vo\(M g ) = l}, 



(2.3) 



where [go]k is defined in (1.8), and 




(2.4) 




Theorem 2.1. Assume 2 < k < ^ and i/ie conditions (CI) (C2) hold. Then the k- 
Yamabe problem (1.1) is solvable. 

As indicated in the introduction, the proof of Theorem 2.1 is divided into two parts. 
The first part is the following lemma. 

Lemma 2.1. If the critical inequality (1.7) holds, then the k-Yamabe problem (1.1) is 
solvable. 

The second part provides the condition for (1.7). 

Lemma 2.2. The critical inequality (1.7) holds for any compact manifold which is not 
conformal to the unit sphere S n . 

When k = t|, we prove that J r n /2(g) = Y n /2{M), that is it is a constant for any 
g G [go]k (Lemma 4.8). Hence (1.7) implies that J r n /2(g) < Yn/2(S n ) provided M is not 
conformal to the unit sphere. 

2.2. Strategy of the proof. A solution of the /c-Yamabe problem is a min-max 
type critical point of the corresponding functional. As we need to restrict ourselves to 
/c-admissible functions, we cannot directly use variational theory (such as the Ekeland 
variational principle). Instead we study a descent gradient flow of the functional and 
investigate its convergence. We need to choose a special gradient flow (similar to [CW2]) 
for which the necessary a priori estimates can be established. 

As with the original Yamabe paper [Ya], we first study the approximating problems 

L(v)=v p , (2.5) 
where 1 < p < Equation (2.5) is the Euler equation of the functional 

J P (v) = J p (v;M) = £±r [ v^- k ^a k (X(V))--±- [ (2.6) 

In - Ak J {M ,g ) P + 1 J(M,g ) 

Let ipi = s and f2 = £~ , where e > is a small constant. Then J p (ip\) — > (when 
k < and J p (<p2) — ► — oo as s — > 0. Let P denote the set of paths in connecting cpi 
and (f2, namely 

P = { 7 e C([0, 1], $ fc ) | 7 (0) = ifu 7 (1) = ^ 2 }. (2.7) 
Obviously &k 0- Denote 



c p [M] = inf sup J p (^(s);M). 
T eP se[o,i] 
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(2.8) 



Then (1.7) is equivalent to 

c p [M] < c p [S n ] (2.9) 

withp = We will prove that J p has a min-max critical point v p with J p (v p ) = c p [A4], 
in the sub-critical case p < By a blow-up argument, we prove furthermore that v p 

converges to a solution of (2.1) under the assumption (2.9). 

The descent gradient flow will be chosen so that appropriate a priori estimates can be 
established. To simplify the computations, we will also use the conformal transformations 
g = u~ 2 go or g = e~ 2w go. That is 

w --2— 

u = e = v ™- 2 

We say u or w is /c-admissible if v is, and also denote u, w G $fc if v E &k- 
Our gradient flow is given by 

F[w]-w t = fi(f(x,w)), (2.11) 

where 

F[w] := fJ,(ak(X(A g ))) (2.12) 

and g = e~ 2w go- When f(x, w) = e~ 2hw , a stationary solution of (2.11) is a solution to 
the fc-Yamabe problem. The function \i is monotone increasing and satisfies 

lim a(t) = -oo. (2.13) 

Condition (2.13) ensures the solution is /c-admissible at any time t. For if u(-,t) is a 
smooth solution, then (2.13) implies o"fc(A) > at any time t > 0. A natural candidate 
for the choice of \x is the logarithm function \x{t) = logt [Chi, Wl, TW4]. However for 
the flow (2.11), we need to choose a different \i to ensure appropriate a priori estimates. 

In the case k = ^, J r n /2(9) is a constant less than Y n /2{S n ). Hence by the Liouville 
theorem in [LL1], it is easy to prove that the set of solutions of (2.1) is compact. Hence 
the existence of solutions can be obtained by a degree argument. When 2 < k < ^, 
by the Liouville theorem in [LL2], one can also prove the set of solutions of (2.5) is 
compact when p < irrf- But to use the condition (1.7) in the blow-up argument, we 
need a solution v p of (2.5) satisfying J p {y p ) = c p . This is the reason for us to employ the 
gradient flow. 

For the verification of (1.7), one cannot mimic the argument in the case k = 1, as the 

test function (1.9) is in general not /c-admissible in a geodesic ball B po . Instead we let 
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(2.10) 



v\ be the solution to the Yamabe problem (k = 1), and v be the /c-admissible solution of 
the equation 



a k (\(V)) = v*- 2 



We will verify (1.7) by using the solution v as the test function. 

The idea of using a gradient flow was inspired by [CW2] , where a similar problem for 
the /c-Hessian problem (see (2.22) below) was studied. However technically the argument 
in this paper is different. For the k- Yamabe problem, the corresponding Sobolev type 
inequality was not available, and the a priori estimates only allow us to get a local (in 
time) solution. The argument in this paper is also self-contained, except we will use the 
Liouville theorem in [LL1, LL2], proved by the moving plane method; see also [CGY3]. 



2.3. The /c-Hessian equation. Equations (2.1) is closely related to the /c-Hessian 
equation 

a k (\(D 2 v)) = f(x) xen, (2.15) 

where 1 < k < n, X = (Ai, • • • , A n ) denote the eigenvalues of the Hessian matrix {D 2 v), 
n is a bounded domain in the Euclidean n-space R n . For later applications we collect 
here some elementary properties of the polynomial a k , and give a very brief summary of 
related results for the equation (2.15). 

We write cro(A) = 1, o~ k (X) = for k > n, and denote cr k -i(X) = <Tfc(A)i _ . 



Lemma 2.3. Let A G T k with Ai > • • • > A n . Then 

Xk > (i 

(7fc(A) = cr k ;i{X) + Xi(Jk-i;i{X), {ii 

E" =1 (T fc _i;i(A) = (n - k + l)<7 fc _i(A), (in 

o- fc _i;„(A) > • • • > <T fc _i;i(A) > 0, (iv 

c/e-i;fe(A) > C njk T^ =1 a k -i-i{X), (v 

**-i(A) > n _l + 1 (t) 1/k M\)] {k - 1)/k . (vi 



Moreover, the function [ak] 1 ^ is concave on T k . 

We just listed a few basic formulae, there are many other useful ones, see for example 

[CNS, CW2, LT]. For our investigation of equation (2.1) and its parabolic counterpart, 

Lemma 2.1 will be sufficient. These formulas can be extended to a k (X(r)), regarded as 

functions of n x n symmetric matrices r. In particular [o"fc(A(r))] 1 / fc is concave in r [CNS]. 
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We say a function v G C 2 (0) is /c-admissible (relative to equation (2.15)) if the eigen- 
values \(D 2 v) G IV Equation (2.15) is elliptic if v is /c-admissible. The existence 
of fc-admissible solutions to the Dirichlet problem for (2.15) was proved by Caffarelli- 
Nirenberg-Spruck [CNS], see also Ivochkina [I]. 

Relevant to the /c-Yamabe problem is the variational property of the /c-Hessian equa- 
tion (2.15), investigated in [CW2, TW4, Wl]. It is well known that the /c-Hessian 
equation is the Euler equation of the functional 

Ik ( v ) = kTl J n (-v)°k(KD 2 v)). (2.16) 

The Sobolev-Poincare type inequality, for /c-admissible functions vanishing on the bound- 
ary, 

lj /{l+1 \v)<Clt /{k+1 \v), (2.17) 

was established in [Wl] for the case / = and k > 1, and in [TW4] for the case k > I > 1, 
where < / < k < n, 

Io(v)=[ [ \vfdx] 1/k \ (2.18) 
Jo, 

and 

k* = n(k + l)/(n - 2k) if k < ra/2, (2.19) 
k* < oo if k = n/2, 
k* = oo if k > n/2. 

The best constant in the inequality (2.17) is attained by 

v{x) = {l + \x\ 2 ) {2k - n)/2k (2.20) 

when I = 0, k < and O = R n ; and by the unique solution of 

— (X(D 2 v)) = 1 in O (2.21) 
o~l 

for 1 < / < k < n. 

^From the inequalities (2.17), it was proved in [CW2] that the Dirichlet problem 

a k (X(D 2 v)) = \v\ p + f(v) in O, (2.22) 
v = on 90, 

admits a nonzero /c-admissible solution, where 1 < k < ^, 1 < p < k* — 1, / is a lower 
order term of \v\ p . The existence result was proved for the problem with more general 
right hand side. 



Remark 2.1. The 1996 preprint [CW1] also contains the existence of solutions to the 
Dirichlet problem (2.22) in the critical growth case p = k* — 1. The result was obtained 
by a blow-up argument and the symmetrization of functions, see Theorem 9.1 in [CW1]. 



2.4. A necessary and sufficient condition for an equation to be variational. 

The following proposition was communicated to the authors by Kaiseng Chou several 
years ago. 

Proposition 2.1. Let M. be a compact manifold without boundary, v G C 4 (A4). An 
operator F[v] = F[V 2 v, Vi>, v, x] is variational if and only if its linearized operator is 
self-adjoint. The functional is given by 



Jo 

This proposition can be found in [O]. We give a proof of the "if part, as we need 
some related formulae. 

Proof. The linearized operator of F[v] is given by 




(2.23) 



except when F is homogeneous of degree -1, where 




(2.24) 



L(<p) = F 13 ^ + Fp.cpj + F v ip. 



(2.25) 



We have 




L 
L 



M 



M 



ip[F tJ Vij + F Pi Vi + F v v]-A + B 



ipL(v) -A + B, 



where 




B 



[ vMVjF'i -F Pi ), 



Jm 



-A + B 



-/ {%v\V 3 F^ - F Pi ) 

JM v 

I ^ V ^ 2 (V^'-F Pi )] 

JM v 
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Hence L is self-adjoint if and only if 



V,[^ 2 (V J F^-F Pi )]=0. (2.26) 



If L is self-adjoint, 



(/>],¥>)=/ ¥> / F[Xv)+ f [ \v[F^[\v\ Vij + F PiVi + F v ip] 

J M JO JO J M 

= f <P f F[Xv}+ f [ \<p[F i i[\v]v ij + F p .v i + F v v] 

J M JO JO J M 

f F[Xv] + f <P [ X^-F[Xv]dX 
o J m Jo dX 



M 

= [ <pF[v]. 

JM 

Hence F is the Euler equation of the functional I. □ 

Conversely if the operator F is the Euler operator of the functional /, from the above 
argument we must have — A + B = 0, namely (2.26) holds. In other words, F is the Euler 
operator of / if and only if (2.26) holds. Observe that if 

Y^ViF ij = F p . Vj, (2.27) 

i 

then (2.26) holds. 

^From Proposition (2.5) we can recover the results on the variational structure of (2.1) 
in [VI]. First, if locally M. is Euclidean, one verifies directly that (2.26) holds, as it is 
a pointwise condition. The locally conformally flat case is equivalent to the Euclidean 
case by a conformal deformation to the Euclidean metric. Finally if k = 2, we note 
that to verify (2.26) for arbitrary v with a fixed background metric go is equivalent to 
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verify it for v = 1 with respect to an arbitrary conformal metric g = v n ~ 2 go- However 
when v = 1, condition (2.26) becomes £™ 7=1 V l V J F^ = 0, where F ij = g^-a k (X(r)) 



at r = A g . But we have 



VtFlJ = 2(^2) (R ' J ~ 2Rij ' i] = ° (2 ' 28) 
by the second Bianchi identity. 

When k > 3, it is easy to find metrics for which (2.26) does not hold (at v = 1). So 
equation (2.1) need not be variational. As an example, let k = n = 3, and in a local 
coordinate system, let the metric g = {gij} be given by 



9u = 1, 922 = 1 + x, #33 = 1 + y 2 + z 2 , g i:j = <% for i ^ j. (2.29) 
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/,jDEFANGED. 171172 Then V^-i™ ^ 0. 

By (2.23) we also see that (2.6) is the functional of (2.5). When fc = §, the integral 
(2.24) may not exist. We may consider v as a composite function v = <p(w) and write 
equation (2.5) in the form 

F[w] =: (p'(w)L((p(w)) = (p p (w)(p'(w). (2.30) 

If the operator L in (2.1) satisfies (2.26) with respect to v, the operator F in (2.30) 
satisfies (2.26) with respect to w. Hence the corresponding functional is given by 

£ n /2(w) = [ [ wF[tw). (2.31) 

J(M,9o) JO 

n — 2 

In particular if v = e ~ w : then we obtain the functional in [BV], 

£ n /2(w) = - [ [ wa n/2 (X(A gt )), (2.32) 

J(M,g ) JO 

where g t = e~ 2tw g . 
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3. The a priori estimates 



In this section we study the regularity of /c-admissible solutions (2 < k < n) to equation 
(2.1) and its parabolic counterpart (2.11). The global a priori estimates for the elliptic 
equation (2.1) (for solutions with eigenvalues in ]?&) have already been established by J. 
Viaclovsky [V2], with interior estimates by P. Guan and G. Wang [GW1]. We will provide 
a simpler proof for the elliptic equation (2.1), and extend the estimates to the parabolic 
equation (2.11) on general manifolds, which is necessary for our proof of Theorem 2.1. 
Previously the estimates for the parabolic equation were proved on locally conformally 
flat manifolds [Ye, GW2, GW3]. Regularity has also been studied in many other papers 
[CGY1,LL1]. 

We will also present an example showing that the interior a priori estimates do not 
hold for solutions with eigenvalues in the negative cone — IV 

3.1. A priori estimates for equation (2.1). For the regularity of (2.1), we will 
use the conformal changes g = u~ 2 go. For function -u, equation (2.1) becomes 

a k (X(U))=u-\ (3.1) 

where 



2 

JJ = V u ^— go + uA go . 



2 



Lemma 3.1. [GW1] Let u G C 3 be a k- admissible positive solution of (3.1) in a geodesic 
ball B r (0) C M. Suppose A go = (a^) G C 1 (S r (0)). Then we have 



|Vd 



u 



(0) < C, (3.2) 



where C depends only on n,k, r, iniu, and \\A go \\c^-, V denotes the covariant derivative 
with respect to the initial metric go. 

Proof. Let \x be a smooth, monotone increasing function. Write equation (3.1) in the 
form 

F[u]=n[f(x,u)], (3.3) 

where F[u] — //[cr fc (A(L r ))]. We will prove (3.2) for more general function /. Moreover the 
constant C is independent of sup Br u if / = ku~ p for some constant p > and smooth, 
positive function k. 

Let z = \Vu\ 2 ip 2 (u)p 2 , where (p(u) = ^, and p(x) = (1 — ^-) + is a cut-off function, 

\x\ denotes the geodesic distance from 0. Suppose z attains maximum at xq G -Bi(O), 
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and |Vu(xo)| = ui(xq). Then at xo, in an orthonormal frame, 

-(logz)i = — + — m + — = 0, (3.4 

2 u± <p p 

-(logz)y = ^ + > ^ H My + 2")«i«j + — / )• 

2 J wi ^ uf ii? v? 2 P p (3.5) 

Differentiating equation (3.3) gives 

F ij [u ij , + (^-^)S ij ] = A, (3.6) 

where for a matrix r = (r y ), F* 3 (r) = ^/i[a k (X(r))] = /i' -£-a k (\(r)), 

A = WMf)-F i ^ 1 (a ij u). 

By (3.4)-(3.6) we have, at xq, 

o > >(iog,)„ = - |t^+ £ ^ + -)(-% + -) 

2 J ui u 2-u 2 ^ uf if p <p p 

+ —F^(uij - ^ Sij ) + u l'^ + (^ - <)F»u5 +F«(£a - ««) + A + A', 
<p 2u 2u (p (p <p l p p z Ui 

where T = F l \ A' arises in the exchange of derivatives, with |A'| < CT . Note that 

F ij ( Uij - ^Sij) = kp,'a k {\) - uFVau > -C a uT, 

where C a = if A go = (a^) = 0. By (3.4) and since (p(u) = ^, 

uii _ _u|_^ + _ uxpx 

u 2u 2 2u (p up 

_ F - J( ^, + £<)(✓ + £) + (!£- = + m.) 

(p p <p p <P <P <PP P 



Hence we obtain 



>Y^ F iJ^?i_ c{ J_ + ^_L + Ca) f + A + A'. (3.7) 
uf r z p z u rp ux 



Denote 6 = ^£-(x ). We claim 



F lJ -u Ql w QJ > Cfe 2 ^ - CVjF (3.8) 



a>l 
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for some positive constant C,C (C = if = 0). Note that by Lemma 2.3 (iii) (vi), 
T > C n , k n'a { k k ~ 1)/k . From (3.8) we have 

IVUI - P <^+C 2 (3.9) 



u r 

at xo, where C\ is independent of / and C<i independent of r. Hence z(0) < z(xq) < C, 
namely (3.2) holds. 

Denote Uij = Uij + uciij. For any two unit vectors £,?7, we denote formally u^ v = 
^drjjUij. Then to prove (3.8) it suffices to prove 

A=:J2 F lJ u ai u aj > Cb 2 T. (3.10) 

OL>\ 

By a rotation of the coordinates we suppose is diagonal at xq. Then 

Ai = mh - b, ■ ■ ■ , A n = u nn - b 

are the eigenvalues of the matrix {uij — <%}• Suppose Ai > • • • > A n . At xq we 
have |Dw(xo)| = u^(xq) for some unit vector £. In the new coordinates we have 

i 

If there exists a small 5q > such that (ej,£) < 1 — 5o for all unit axial vectors e^, 
then A > SqF^u^. Since A = (Ai, • • • , A n ) G T^, we have Afc > and so Ukk > b. Hence 
by Lemma 2.3(v), A > 5 b 2 F kk > b x b 2 T. We obtain (3.8). 

If there is i* such that (ej*,£) > 1 — do, then we have A > \ Yli^i* F 11 ^. If there 
exists j > k, j ^ i* such that ujj > ah for some a > 0, then by Lemma 2.3(iv)(v), 
A > \F^{ab) 2 > b 2 b 2 T and the claim holds. Otherwise we have i* = k since Ukk = 
X k + b>b. 

Case 1: k < n — 2. Observing that • • • d ^ ^ ^(A) = Afc + ■ ■ • + A n > 0, we have 
Afc > — (Afc+i + • ■ ■ + A n ). Since Ujj < ab for j > k + 1, we have Aj < —(1 — a)b. Hence 
A fc > (n - k)(l - a)b > 2(1 - a)b. 

On the other hand, by (3.4), we may suppose at x , \^-\ < a^-, for otherwise we 
have the required estimate (3.2). Hence < (2 + a)b for a different small a > 0. 
By the relation u# = ^i^i^u > J2i<k Zi^a ~ na ^ where £ = (£i,--- ,£ n ), we have 
w/cfc < (l + a)u££ < (2 + 2a)6. Hence Afc = Ukk — b< (l + 2a)6. We reach a contradiction 
when a is sufficiently small. 

Case 2: k = n — 1. We have 

dofc , /x s Ai---A n Ai • ■ • A n 
Afc-i = (Tfc^Aj > 



dXk-i Afc_i Afc 

14 



Since X n = u nn — b < — (1 — a)b and by • • • dx d - <7fc(A) = A n _i + A n > 0, we have 
A n -i > (1 — and so Aj > (1 — a)b for any 1 < i < n — 1. Hence 9 ^ k l Xk-i > 
(1 - a) 2 6 2 Ai • • • A n _ 3 . Note that v'%^{X) = F u . It follows that 

1 , dak ~9 1 / 9<7k ,9 / \ 

A > — u ttt %_i i > -u — Ai,_i (3.11) 

~ 2 P aA fc _i fc lfc 1 - 2 P 9A fc _i fc 1 v ' 

> ^'{l-afb 2 X 1 ---\ n -2>Cb 2 T. 

Case 3: k = n. As in Case 1, we assume that |^-| < a^-. Then by (3.4), > (2—a)b. 
Note that when k = n, A^ > for all i. Hence ua = Xi + b > b. Recall that when k = n, 
we have i* = n. It follows that u nn > (2 — a)b for a different small a > 0. Hence 
A n > (1 — «)fr and 

A > hl^F^ul > X -F^X\ = ^XiX n F nn > Cb 2 T. (3.12) 

This completes the proof. □ 

We remark that in our proof of (3.10), we didn't use the equation (3.3). Hence we 
can also use (3.10) for the corresponding equation (3.21) below. We also note that the 
gradient estimate is independent of the choice of fx. From Lemma 3.1, we obtain the 
following Liouville theorem. 

Corollary 3.1. Let u G C 3 be an entire k- admissible positive solution of 

a k (X(V 2 u- l -^-I))=0. (3.13) 

Then u = constant. 

Proof. For equation (3.13), the constant C 2 in (3.9) vanishes. Letting r — > oo, by (3.9), 
we see that either = 0, or T = 0. In the former case, u is a constant. In the latter 
case, u satisfies ak-i(X) = and so it is also a constant by induction. □ 

By approximation as in [MTW,W2] one can show that Corollary 3.1 holds for contin- 
uous positive viscosity solutions. The proof of the interior gradient estimate (3.2) can 
be simplified if one allows the estimate to depend on both mfB(o, r )U and sup B( - ^ u. 
Indeed, let (p(u) = in the auxiliary function z, where S = |inf B( - r )W. Then one 

obtains the extra good term ^_^ m2 JF on the right hand side of (3.7). The proof after 
(3.8) is not needed. 

For the /c-Yamabe problem, f(u) = u~ k . The constant C in (3.2) is independent of 

sup u. Therefore we have the Harnack inequality [GW1] . 
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Corollary 3.2. Let u E C 3 be a positive solution of (3.1). If inf u > C > 0, then 
sup u <C\. 



Next we prove the second order derivative estimate. 

Lemma 3.2. Let u G C 4 be a k- admissible positive solution of (3.1) in a geodesic ball 
B r (0) C M. Suppose A e C 2 (B r (0)). Then we have 

|V 2 w|(0)<C, (3.14) 

where C depends only on n,k, r, inf it, suptt, and \\A go \\ C 2. 

Proof. Again we will consider the more general equation (3.3). Choose «(t) = t x l k such 
that equation (3.3) is concave in Uij. Differentiating (3.3) we get 

/••"'•,,.,, = - ^ff w,^ + v 2 ,M/) > V 2 u(/), (3.15) 

where Uij^k = ^kUij. As above denote Uij = + ua^. Let T denote the unit tangent 

bundle of B r (0) with respect to go. Assume the auxiliary function z on T, z(x,e p ) = 

p 2 \7 2 u(e pi e p ), attains its maximum at xq and in direction e\ = (1,0, ••• ,0), where 
i i 2 

p(x) = (1 — ■^2-) + . In an orthonormal frame at xq, we may assume by a rotation of axes 
that {Uij} is diagonal at xq. Then at xo, F lJ is diagonal and 

0=(log^ = ^ + ^, (3.16) 
P 

> (logz)« = { ^-) + — • (3-17) 

P P 2 un 

By (3.16), the gradient estimate, and the Ricci identities, 



U ijA1 = Uijll - &6 tj + 0(i±M) = u luj - ^ + 0(i±^i). (3.18) 



Hence we obtain 



> VF«(logz)« > 

~ p 2 Mn 

>-^+^^+— v 2 M/)- 

Since fi(t) = we have T > C > 0. Hence (3.14) holds. □ 

The second order derivative estimate (3.14) was established in [GW1]. As the proof 
is straightforward, we included it here for completeness. The estimate is also similar to 
that in [GW4] for the equation 

det(V 2 w- ^^1 + ^i) = f(x,u,Vu) in OcT (3.19) 
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which arises in the design of a reflector antenna, where I is the unit matrix. See also 
[W2] for n = 2. 

By Lemma 3.2, equation (3.1) becomes a uniformly elliptic equation. By the Evans- 
Krylov estimates and linear theory [GT], we have the following interior estimates. 

Theorem 3.1. Let u G C 3,1 be a positive solution of (3.1) in a geodesic ball B r (0) C A4. 
Suppose f > 0, G C 1 ' 1 . Then for any a G (0, 1), 

IMIc3.«(B r/2 (o)) < C, (3.20) 

where C depends only on n, k, r, inf_A4 u, and go. 

Theorem 3.1 also holds for equation (3.3) with / = ku~ p for a constant p > and a 
smooth, positive function k. 



3.2. The parabolic equation. It is more convenient to study the parabolic equation 
for the function w = log it. In this section we will extend the a priori estimates in §3.1 
to the parabolic equation 

F[w]-w t = n(f), (3.21) 



where F[w] = /z[<7fc(A(W))], and 



W = V 2 w + Vw®Vw - ^|Vio| 2 fir + A go . 



When f = e 2kw , a stationary solution of (3.21) satisfies the equation 

a k (X(W)) = e~ 2hw , 

which is equivalent to (3.1). 

We choose a monotone increasing function \i such that F is concave in D 2 w and 

t x l k t > 10, 
logt te(0,±), 



n(t) = 

and furthermore 



(t - s)(»(t) - n(s)) > co(t - s)(t'' k - 8 L ' K ) (3.22) 

for some constant cq > independent of t. Condition (3.22) will be used in the next 
section. 

We say w is /c-admissible if for any fixed t, w is fc-admissible as a function of x. Denote 
Q r = B r (0) x (0, r 2 ]. In the following lemmas we establish interior (in both time and 
spatial variables) a priori estimates for w, 
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Lemma 3.3. Let w be a k- admissible solution of (3.21) on Q r . Then we have the 
estimates 

|V x «;(0,r 2 )| <C, (3.23) 

where C is independent of sup w, if f = K,(x)e~ pw for some constant p > and smooth, 
positive function k. 

Proof The proof is similar to that of Lemma 3.1. We outline the proof here. Let u = e w . 
Then u satisfies the equation 

- ^ = /*(/), (3.24) 
u 

where 

1 iVwl 2 

F M = ^[^k a k{\(v 2 u - H^-go + uA go ))]. 

Let z = (^^■) 2 p 2 be the auxiliary function as in the proof of Lemma 3.1. Here we choose 

p(x,t) = ±(l- l -^) + . 

Suppose z attains its maximum at (xo, to). Then to > 0. By a rotation of axes we assume 
|Vit| = u\. Then at (x 0l t ) : Z{ = 0, {%} < 0, and z t > 0. Hence we have (3.4), (3.5) 
and 

^-^ + ^>0. (3.25) 
Differentiating equation (3.24) we obtain (3.6) with F y and A replaced by 

Hi Uj (Jh 

> -^ + V 1 p(f)-F^V 1 (a lJ u). 
By Lemma 2.3 and our choice of p, T = F" has a positive lower bound, 

r>^ {u ~ k l k{x) Kt iy \Ku))>- 

u k u 

for some C > depending only on n, k. Hence similarly as the proof of Lemma 3.1 we 
have (3.7). From (3.8), we obtain estimate (3.23). □ 

Since the constant C in (3.23) is independent of sup w, hence we have a similar Harnack 
type inequality as Corollary 3.2. 
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Lemma 3.4. Let w be a k- admissible solution of (3.21) on Q r . Then we have the 
estimate 

\V 2 x w(0,r 2 )\<C, (3.26) 
where C depends only on n,k, r, p, inf w, suptu, and ||-A ff0 ||c 2 - 
Proof. Differentiating equation (3.21) twice, we get 

F ij W ij , k =w tk + V kf JL(f), (3.27) 
F^W ijM = /••"•' H I •;,. / ,U V.j, + w tkk + Vlfi(f) > w tkk + V 2 p(f). 

(3.28) 



w here F ij = dF W ■ u = ViW • and F ij > rs = 9 ^ ak( - x( - w ^ Denote w - = w - + a- ■ 
aij = (A go )ij. Let T denote the unit tangent bundle of Ai with respect to go- Consider 
the auxiliary function z defined on T x [0,r 2 ], given by z = p 2 (y 2 w + (Vw) 2 )(e p ,e p ), 
where p is the cut-off function in the proof of Lemma 3.3. Assume that z attains its 
maximum at (xo,to) and in direction e\ = (1,0, ••■ ,0). We choose an orthonormal 
frame at (xo,to) 5 such that after a rotation of axes, {M^j} is diagonal. Then i™ is 
diagonal and at (xo,to) 5 

0=(logz)i = H h : 2 ' 3 - 29 

p wn + wf 

0<(log2)t = H z ; 2 — ' 3 - 30 

p wn + wf 

0>(logz)ii = ( 2") H : = ■ 2 • ( 3 - 31 ) 

p p 2 Wn + wf 

We have, by (3.29) and the Ricci identities, 

Wij,n = + ^ai^j + wjuWi + 2wnWji - w 2 kl 5 i:j + 0(^(wu + w 2 )) 
= wmj + 2wnWji - w kl Sij + 0(^(wn + w 2 )). 
Hence we obtain 

0> ^"(logz^-Oogz)* 

i 

^ C t, 1 ™/~ , ,o 2n w llt + 2w 1 w lt 2p t 

> ; n F {wil,ii + 2w 1 W iil +2w il ) z ; o 

p 1 wn+Wi wn+wf p 

> 2^+" 1 2 F [( W H,11 + W kl) +2WiW Hi = ■ 2 

^~^ :F + ~ \ 2 ( FiiW HM - W Ht) + WllF + ~ ^ 2 ( Fiiw m - W tl) - — 

p 2 wn+wf wn+wf p 

>~^+ - \ g Vg/*(/) + U7H.F + . ^ 2 ViM/) - — ■ (3-32) 
wn+wf wn+wf p 
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By our choice of fi, T > C for some C depending only on n, k. We obtain wnp 2 < C at 
(x ,t ). Whence z(0, r 2 ) < z(x ,t ) < C. □ 

Remark 3.1. The a priori estimates (3.23) and (3.26) also hold for the equation 

-/j(a k a(X(W)) -w t = -»(a k f), (3.33) 

(X Ct 

where a > is a constant. We claim that the constants C in (3.23) and (3.26) are 
independent of a > 1. 

First we note that this is obvious for (3.23) as the gradient estimate is independent of 
the choice of p. 

For the estimate (3.26), we have, by Lemma 2.3, 

1 



p{a k a k {\{W)) 



a 



= (n- A; + l)a fc -V fc _i(A(W))^ 

> Ca fc "Vf - 1}/ V(a fc ^) 

> cmf^-^/ym, 

t>o 

By our choice of //, inf t>0 *( fc_1 >/ V(*) > C > 0. Hence ^ > C > 0. 
Therefore by (3.32) it suffices to show that 

|Vi<7| + \Vjg\ < C 

for some C > independent of a > 1, where g = ^fi(a k f). By our choice of 

M/) if a*/ > 10 

i(/cloga + log/) ifa fc /<^, 

Hence supQVigl + IV 2 ^]) is independent of a > 1 if a k f > 10 or a k f < j^. When 
a k f E (jq, 10), we can choose n properly such that sup ^\g\ is independent of a > 1. 
Alternatively one can compute directly 

\v ig \ = i^-V^-^W/)! IfeVi/ 1 /*! 



# = 



< ciVi/ 1 ^!, 

|V?^| = |a fe -V(a fe /)V^/H-|a''- | / /'(> ; '' f) 



where C depends on sup t>0 t 1-1 /*^'^) and sup t>0 < oo. Hence the estimate (3.26) 

is independent of a > 1. 

Remark 3.1 will be used in the next section. The choice of function [i in the parabolic 
equation (3.21) is critical. 
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Lemma 3.5. Let w be a k- admissible solution of (3.21) on Q r . Then we have the 
estimates 

\w t (0,r 2 )\<C, (3.34) 
where C depends only on n,k, r, p, inf w, supiu, and || ^4 9o || c 2 - 

Proof. From the equation (3.21) and by the estimate (3.26) we have an upper bound 
for wt- It suffices to show that wt is bounded from below. Let z = (m-w)" i wnere 
M = 2 supg r | it? |, p is the cut-off function as above. Suppose ming r z attains its minimum 
at (xo,to), t > 0. Then at the point we have z t < 0, Zi = and the matrix {zij} > 0, 
namely 

+ a— +13— >0, (3.35) 

M — w p 

— + a— ^+/3^ = z = l,-..,n, (3.36) 
w t M -w p J 

L w t w 2 t M-w (M-w) 2 p p 2 J v ' 

where we have changed the direction of the inequalities as we assume that wt < 0. 
Differentiating equation (3.21) gives 

F^W ijt -w tt =^(f). (3.38) 



Hence by (3.35), 



Wt . W tt nPt 

a— > p — 

M — w Wt p 



w t w t ot p 



By (3.36), the matrix in (3.37) is equal to 



Wjjt awjj a(l ~ a)wiWj _ 2a[5w i p j p^ _ pipi 
w t M-w (M-w) 2 (M-w)p ' p l { P) p 2 



We have 



— F l3 Wi jt = —F tJ (w ijt + w it Wj + Wj t Wi - w k w k tdij) 
w t w t 

> f«j7 aWi i jl «(! - a)wiWj _ 2af3w i p J p^_ _ p t p t 
~ { M-w (M-w) 2 (M-w)p ' p pyL ^ P) p 2> 

+ /••<-'(2o-^ + 2/3^ - ap^ Slj - 

M — w p M — w P 

> — - — F ij ( Wij + 2wiWj - \Vw\ 2 5 lJ ) 
M — w 

I a jrw ( — ot)wjWj n „w lPj _ C_ 
^ M-w [ M-w ' p ] p 2 " 
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where the constant C depends on the gradient estimate (3.23) and the second derivative 
estimate (3.26). Choose a=\. By the Holder inequality, 



w i w j _ oq W iPi) > 

■2(M-w) p ' ~ p 2 



By the fc-admissibility, F^Wij > 0. Hence we obtain 



— F^W ijt > —^—^(Wij + w iWj - \\Vw\ 2 5 l3 - aij ) -^>~. 
w t M — w 2 J J p 2 p 2 



It follows that 

w t C 1 d p t 

M — w p 2 w t at p 

Now we choose (3 = 2. Then we obtain 

z{xoM) = (M-wy/ 2 p2{x °' to) - ~ a 

It follows that z(0,r 2 ) > z(xo,to) > —C. Hence wt is bounded from below. □ 

Theorem 3.2. For any wo G <&k, there is a smooth k-admissible solution w G C 3,2 (A4 x 
[0,T)) of (3.21) with w(-,0) = wq on a maximal time interval [0, T). IfT< oo, we have 
infx w(-, t) — > — oo as t ST. 

Proof. First we point out that a /c-admissible solution of (3.21) is locally bounded. 
Indeed, at the minimum point of w, by equation (3.21) we have 

w t = F[w] - p(f) > p(a k (X(A go ))) - p(f). 

Hence locally in time the solution is bounded from below. By the interior gradient 
estimate (3.23), the solution is also bounded from above. Therefore by Lemmas 3.3-3.5, 
equation (3.21) is uniformly parabolic. By Krylov's regularity theory, we obtain the C 3 ' 2 
a priori estimate for (3.21), and so the local existence follows. Let [0, T) be the maximal 
time interval for the solution. If T < oo, we must have inf M w(-,t) — > — oo as t / T. □ 

Remark 3.2. The a priori estimates in §3.1 and §3.2 can be extended to the quotient 
equation 

a k ,i(X(V 2 u-^-g Q + uA)) =u l ~ k (1 < / < k < n) (3.39) 

and its parabolic counterpart, where &k,i(\) = ^f(X). Indeed, let p be a monotone 

increasing function such that p[o~k,i(\)] is concave. Write equation (3.39) in the form 

F[u] = p(f) as (3.3). Then the proof for the second derivative estimates (3.14) and (3.26) 
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can be extended to the quotient equation without change. For the gradient estimates 

d 
OK 



(3.2) and (3.23), denote a k -i-i(X) = gx^ fc (A). B ^ Newton's inequality [LT], 



_ a l a k-l;i — &k&l-l-i 
l A J - ~2 



dXi af 



_ Vl-jVk-l-i - Vk;i<?l-l;i njk - /) ai-iCTk-l;i /„ 

-k(n-l) a? ■ {6AU) 

As before we arrange the eigenvalues in the descending order Ai > • • • > A n . Then by 
Lemma 2.3(v), a Z;i (A) > Cai(X) when i>l + l. Hence ^(A) > CJ2i ^Bf( X )i namely 
F kk > CT . Hence the proof of (3.2) and (3.23) can also be extended to the quotient 
equation. But we need to replace (3.11) in Case 2 by 



1 , da k ,l ~ 2 ^ 1 i n ( k - g7;fc-l0fc-l;fc-l _ 2 

2^ dXkT^- 1 '- 1 - 2^ W^T) ^ u 



fc-i fc-i 



and (3.12) in Case 3 by 

A > l^F^ul > l -F u ul > C a -^ul > CX L X n ^^ > Cb 2 f, 
2 2 a\ G\ 

where we have used F u > Ca k -i,l/vi by (3.40). 

For the corresponding parabolic equation (3.21), where F[w] = fj,(ak,i(X(W))), choose 
a monotone increasing function ji such that F is concave in A, and /j,(t) = tVC^-O when 
t > 10, n(t) = \ogt for t > small. Then we can prove Theorem 3.2 for the quotient 
equation in the same way as before. 

We remark that the a priori estimates for (3.39), and for its parabolic counterpart on 
locally conformally flat manifolds, were obtained in [GW3]. 

We also note that the a priori estimates in §3.1 can be extended to the more general 
equation 

fc-i 

s fc = (3.41) 

1=0 

where fii are nonnegative constants, J2i Pi > 0, and s k = u k ak{X(V 2 u — — h uA)) is 
the /c-curvature. 



3.3. Counterexamples. Theorem 3.1 applies to solutions of (3.1) with eigenvalues 

in the positive cone IV The a priori estimate (3.14) relies critically on the negative sign 
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of the term , which yields the dominating term u\ x in (3.18). Equation (3.1) has 
another elliptic branch, namely when the eigenvalues A lie in the negative cone — IV An 
open problem is whether the a priori estimate (3.14) holds for solutions with eigenvalues 
in the negative cone — IV This is also an open problem for equations from optimal 
transportation [MTW], in particular the reflector antenna design problem (3.19). Here 
we give a counter example to the regularity. Our example is a modification of the Heinz- 
Levy counterexample in [Sc]. 

We will consider the two dimensional case. By making the change u — > —u, we consider 
equation 

det(u y + |Vu| 2 / + a y ) = / (3-42) 

with positive sign before the term |V-u| 2 , where / is a C 1,1 positive function to be 
determined. We want to show that equation (3.42) has no interior a priori estimates for 
solutions with eigenvalues in the positive cone. 

Set 

u{x) = -x\ + <pM, (3.43) 
where b is constant, <p is an even function. Let 

an = — b x 2 , a\2 = 0, 022 = —b — b 2 x\. (3.44) 
Then equation (3.42) becomes 

(<p" + v' 2 W 2 = f. (3.45) 
Let ip = ((p') 3 . Then ip satisfies the equation 

Let 9 

ip(xi) =xx- -xl /3 . 

Then 

m = l-xi /a +(xi-*xl /3 )*' 3 

is a positive C 2 function, but the solution u £ C 2 . 
If instead of (3.44), we choose 

011 = Co — b 2 x\, 0,12 = 0, a,22 = e — b — b 2 x\, (3.49) 

where Co, £ are constants, e > small. Then we have the equation 

(<p" +^' 2 + c )(e + ^ 2 )=f. (3.50) 
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(3.46) 

(3.47) 
(3.48) 



Let / = 1 and denote g = cp'. Then g(0) = and g satisfies 




1 



- g 2 - c - 



(3.51) 



e + g 2 



This equation has a unique solution g e . Obviously the gradient of g e is not uniformly 
bounded. Hence there is no interior C 1 ' 1 a priori estimate for equation (3.43). Note that 
the matrix A = (a^) can either be in the positive cone or in negative cone by choosing 
proper constants 6, cq- 

Write equation (3.1) in the form 



Then similarly as above we can construct a sequence of functions satisfying equation 
(3.52) with / = 1 whose second derivatives are not uniformly bounded. 

Remark 3.3. In many situations [MTW, W2] there arise equations of the form 



where A is a matrix. From the discussions in this section, we see that the interior a priori 
estimates hold in general when A is negative definite with respect to Du, and do not 
hold if A is positive definite. When A = 0, there is no interior regularity in general, but 
if the solution vanishes on the boundary, interior a priori estimates have been established 




(3.52) 



a k (\(D 2 u + A(x, u, Du)) = /, 



(3.53) 



in [CW2]. 
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4. Proof of Lemma 2.1 



4.1. Existence of solutions in the sub-critical growth case. In this subsection 
we first study the existence of /c-admissible solutions, for 2 < k < l|, to equation (2.5) 
in the subcritical growth case 1 < p < We then extend the existence result to the 

critical case p = in §4.2 by the blow-up argument. In §4.3 we consider the case 
fe = §. 

Theorem 4.1. Suppose 2 < k < ^. Then for any given 1 < p < ^35, there is a solution 
v p of (2.5) with Jp(v p ) = c p > 0, where J p , c p are defined respectively in (2.6) and (2.8). 
Moreover, the set of solutions of (2.5) is compact. 

A solution of (2.5) is a critical point of the functional J = J p . To study the critical 
points of the functional J, we will employ the parabolic equation (3.21), which can also 
be written in terms of v as (ignoring a coefficient before vt) 

F[v) + ^ = v(f(v)), (4.1) 
where f(v) = v~^~ e , F[v] = /^(cr fc (A(^))), fx is the function in (3.21), and 

Write functional (2.6) in the form 

J(v) = tt I v ™- 2 cr k (\( — )) / 

Equation (4.1) is a descent gradient flow of the functional J, 

d T/ \ f 2n-4fe_ 1 , W V\. -*»S--e1 

— J(v)= u «-» [a k (X(—))-v n - 2 ]v t 

dt J(M,o n ) V 



2n-4fe 4fe 



v ™- 2 v n ~ 2 £ . (4.3) 

(M,g ) v Pti^) 



(M, 90 ) 

I" 2n-4fc r / ^ / V s s 4fc r , , W V 

J(. 



v ^[a k {\{-))-v—*- s ] [fu(a k (X(-))) - fu(v^- e )] < 0. 



Given an initial fc-admissible function vq, by Theorem 3.2, the flow (4.1) has a unique 
smooth positive solution v on a maximal time interval [0, T), where T < 00. 

Lemma 4.1. Suppose J(v(-,t)) is bounded from below for all t G (0, T). If v(-,t) is 
uniformly bounded, then either v(-,t) — > or i/iere is a sequence tj — > 00 swc/i £/iai 
v(-,tj) converges to a solution of (2.5). 
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Proof. By the assumption that v(-,t) is uniformly bounded, we have T = oo. At the 
maximum point of v(-,t), by equation (4.1) we have 

v t <v[^f(v))-f,(a k (X(A go )))}. (4.5) 

Hence if sup v(-, to) is sufficiently small at some to, by the assumptions go G Tfc and v > 0, 
we have i>(-, t) — > uniformly. Therefore if i> does not converges to zero uniformly, by the 
gradient estimate (3.23), we have v > c for some constant c > 0. In the latter case, by 
Theorem 3.2 and the assumption that v is uniformly bounded, we have \\v(-, t)||c 3 (A4) < C 
for any £ > 0. 

Choose a sequence tj — > oo such that 

Ij^^-^O. (4.6) 

By the above C 3 a priori estimate, we may abstract a subsequence, still denoted as tj, 
such that v(-,tj) converges in C 2 ' a . By (4.4) we concludes that v(x,tj) converges as 
j — > oo to a solution of (2.5). □ 

Lemma 4.2. Suppose J(v(-,t)) is bounded from below for all t G (0, T). TTien T = oo 
andv(-,t) is uniformly bounded. 

Proof. Suppose to the contrary that there exists a sequence tj / T such that rrij = 
sup v(-,tj) — > oo. Assume the maximum is attained at Zj G M.. By choosing a normal 
coordinate centered at Zj, we may identify a neighbourhood of 2^ in M. with the unit 
ball in R n such that Zj becomes the origin. We make the local transformation 

Vj(y, s) = m~ 1 v(x,t), (4.7) 

2 g 

y = m j~ 2 2 " x ' 



4 



s = m ;- 2 *(£-£,). 

For the transformation x — > y, more precisely it should be understood as a dilation of 
M., regarded as a submanifold in R^ for some N > n with induced metric. Denote 
Mj = {Y = mJ^~*X | X G M C R^}, with induced metric from R N . Then we have 
< Vj(y,0) < m~ 1 v(0,tj) = 1, Vj is defined for y G Mj and s < s , where by (4.5), 
so > is a positive constant independent of j. Moreover Vj satisfies the equation 



J 3 

4 I e 4fc V". 4 i e 4fc 



rrij ■ - >[m;- 2 a fc (A(-^))] +^ = m,- - 2 "VK" 2 /(«,-)). (4.8) 



By direct computation, 

/ i; J --s=5- < 7 fc (A(^))dy = mf 1 2fe) / cx fc (A(-))efe (4.9) 

•/.VI , ./.V! V 

2n 



/ 



Vj ™- 2 dy = m- j v n ~ 2 ax 



M 
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Hence 



J(vj,Mj) 



n-2 
2n — 4k J M 



V* 



vj ™- 2 (T k (X(—))dy 



'J(v,M) < C. 
We may choose Sj £ (0, |sq) such that 



■dy 



(4.10) 



-J( Vj (., Sj ))^0. 



By (4.4), (4.11) is equivalent to 



/ vj" k L k (\&)) - Vj 



4k 



Tfli 



/.(m;- 2 V fc (A(-^)))-/*(m; 



4fc 4fc 
-2 %,™-2 £ N 



By (3.22), we obtain 



K(A(^))) 1/fe -(^- £ ) 1/fc 



(4.11) 



0. (4.12) 



By the gradient estimate (3.23), Vj + ^- (at s = Sj) is locally uniformly bounded. Hence 



in L( fe+1 )/ fc . 



(4.13) 



Note that by Remark 3.1, vj are locally uniformly bounded in C 1,1 and the convergence 
in (4.13) is locally uniform. 

By extracting a subsequence we can assume that Vj(-,Sj) converges to a function 
vq £ C 1 ' 1 (R n ) with vq(0) = 1. We claim that vq is a smooth solution of the equation 



F [v] :=al /k (\(V))=v? 



(4.14) 



in R n . On the other hand, by the Liouville Theorem in [LL2], there is no entire positive 
solution to (4.14) when e > 0. This is a contradiction. Hence Lemma 4.2 holds. 

To prove that v is a smooth solution of (4.14), we note that since v G C 1 ' 1 (R n ), v 
is twice differentiable almost everywhere. Suppose now that Fo[^o] > Vq at some point 
x where v is twice differentiable. Without loss of generality we assume that x = 0. Let 

1 e 
(f{x) = v (0) + Dv (0)x + -DijVQ^XiXj + - \x\ 2 - 5, 
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where e, S are positive constants. By choosing S sufficiently small we have 

(p > v o on dB r (0) and (p(0) < i>o(0). 

Since i^- — > i>o uniformly, we have </? > i>j on dB r (0) and </?(0) < f j(0) when j is sufficiently 
large. Since vo is locally uniformly bounded in C 1,1 , by the inequality F [vo] > Vq we 
have X(V Vo — el) G and 

F M : = ^[A(-VV + ^^-^ " — 2 ^« + > < 

when e > is sufficiently small, where V Vo is the matrix relative to vq, given in (2.2). 

1 / k 

Hence by the concavity of a k ' , 

F ab [ Vj ]D ab (<p - V j) < F( Vj ) - 1£ - (4.15) 

in LP(Q) for any p < oo, where F ab [vj] = -^—o^ ^ \\{f)) at r = V v . (a, 6 = 1, • • • ,n), 
which satisfy detF ab > C > for some C > depending only on n, k. Applying 
the Aleksandrov-Bakelman maximum principle [GT] to (4.15) in {(p < vj} and sending 
j — > oo, we conclude that cp > vo near 0, which is a contradiction so that Fo[vo] < Vq at 
xq. By a similar argument we obtain the reverse inequality and hence we conclude (4.14) 
for vo. Since the limit equation (4.14) is locally uniformly elliptic with respect to vq, we 
then conclude further regularity by the Evans-Krylov estimates and linear theory [GT]. 
In particular we obtain vq G C°°. □ 

A more general approach to the approximation argument to obtain (4.14) can be 
obtained by extending the theory of Hessian measures in [TW1, TW3] to the operators 
of the type F . 

Lemma 4.3. There exists a function vo G $jt such that the solution v of (4-1) satisfies 
J{v{-, t)) > —C and supv (•, t) > cq > for all t > 0. 

Proof. Let P be the set of paths introduced in §2.2. For 7 G P, let v s (s G [0, 1]) be the 
solution of (4.1) with initial condition v s (-, 0) = 'j(s). Then by (4.5) and the comparison 
principle, there is an sq > such that v s (-,t) — > uniformly for s < sq. Denote by 
I 1 the set of s G [0,1] such that J(v s (-,t)) > for all t > 0. Then (0, so) C I 1 . Let 
s* = sup{s I s G I 7 }. 

Obviously s* G I 7 . For if there exists t such that J(v 8 *(-,t)) < 0, then J(v s (-,t)) < 
for s < s* sufficiently close to s*, which implies s* 7^ sup{s | s G I 7 }. It is also easy to see 
that v s *(-,t) does not converges to zero uniformly, for otherwise v s (-,t) — > uniformly 
for s > s* and near s*. Finally by our definition of the set P, we have 1 ^ J 7 , namely 
s* < 1. Hence vq = 7(5*) satisfies Lemma 4.3. □ 

^From the above three Lemmas, one sees that there is a sequence tj — > 00 such that 

v 8 *(-,tj) converges to a solution of (2.5) for 1 < p < Next we prove 
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Lemma 4.4. For any given 1 < p < the set of solutions of (2.5) is compact. 

Proof. By the a priori estimates it suffices to show that the set of solutions is uniformly 
bounded. If on the contrary that there is a sequence of solutions vj G such that 
sup^j — > oo, denote rrij = swpvj and assume that the sup is attained at Zj. Similar 
to (4.7) we make a translation and a dilation of coordinates and a scaling for solution, 
namely 

Vj(y) = mj^jix), 

y = RjX Rj=mf^~^. 

Then < Vj < 1, and Vj satisfies 

a k (X(V))=v k ^- £ . 

By the a priori estimates in §3.1, v is locally uniformly bounded in C 3 . Hence Vj converges 
by a subsequence to a positive solution v of 

a k (X(V)) = v k ^~ £ in R n . (4.16) 

By the Liouville Theorem [LL2] , there is no nonzero solution to the above equation. We 
reach a contradiction. □ 

Let v be a /c-admissible solution of (2.5). Then we have 

I v ™- 2 a k (X( — )) - / V ™" 2 V"- 2 =0. 

J(M,g ) v J(M,g ) 



Hence 



By (4.3) we have 



J(v) = sup J(tv). 



j / \ , n-2 1 f 

J{V) = (- -) / V ™- 2 V ™- 2 

y) K 2n-4k p+l J J iM , go ) 

> C > (4.17) 

By the compactness in Lemma 4.4, the constant C is bounded away from zero. 
Lemma 4.5. There exists a solution v p of (2.5) such that J(v p ) = c p . 

Proof. For any given constant 5 > 0, choose a path 7 e P such that sup s€ ( 01 ) J{ r ){s)) < 

c p + 5. By the proof of Lemma 4.3, there exists s* G (0, 1) such that the solution of (4.1) 
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with initial condition v(-,t) = "f s * converges to a solution v* s of (2.5). Since (4.1) is a 
descent gradient flow, we have J{vf) < c p + 5. Letting 5 — > 0, by the compactness in 
Lemma 4.4, v* s converges along a subsequence to a solution v of (2.5) with J(v) < c p . 
Note that J(v) — sup s>0 J(sv) > Cp. Hence J(v) — c p . □ 

^From (4.17) we also have 

c p > C > 0. (4.18) 

We have thus proved Theorem 4.1. 

4.2. Proof of Lemma 2.1 (for 2 < k < t|). Let v p be a solution of (2.5) with 
Jp(v p ) = c p . If there is a sequence pj / ^| such that supw Pj is uniformly bounded, by 
the a priori estimate in §3.1, v Pj sub-converges to a solution of (2.1) and Lemma 2.1 is 
proved. 

If supVp — > oo as p /" ^3§, noting that c p < sup s>0 J(svo) for any given vq E we 
see that c p is uniformly bounded from above for p e [1, ^3§]- By (4.17), 

/ < +1 < C, (4.19) 

where C is independent of p < ^3§- Denote m v = supv p and assume that the sup is 
attained at z p = 0. As before we make a dilation of coordinates and a scaling for solution, 
namely 

v P (y) = m~ 1 v p {x), 

2 e 

y = RpX R p = rrip~ 2 2k . 
Then < v p < 1, and v p satisfies 

in B c r p for some constant c > independent of p. Note that in the present case, 
£ = — P ^ 0. By the a priori estimates in §3.1, v is locally uniformly bounded in 
C 3 . Hence v p converges by a subsequence to a positive solution v of 

a fc (A(F) = v k ^ in R n . 

By the Liouville Theorem [LL1], 

v(y) = c(l + \y\ 2 ) 2 -^ 1 , (4.20) 
where c = [n(n — 2)]( n_2 )/ 4 . Moreover 

sup J p «(sw;R n ) = Cp.[S n ], (4.21) 

s>0 
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with p* = where c p was defined in (2.9). 



4 

The above argument implies that the metric g = Vp ~ 2 go is a bubble near the maximum 
point z p . By (4.20), v p has the asymptotical behavior 



5 



v p {x) = c{ w ^)^{l + o{l)) 5 = m p ^ + 2k . (4.22) 

For a sufficiently small 9 > 0, let Vt p = {x G M. \ v p (x) > 9m p }, and let 

v p (x) x E M — O p , 



v p (x) = 

Note that by assumption (1.7), 



9m p x G Vt p . 



sup J(sv p ) = c p < c p * [S n ] (4.23) 

s>0 

when p < and is close to 

Combining (4.21), (4.22) and (4.23), we see that 

/ % +1 > C > 

J(M,g ) 

for some C independent of 6>, provided 9 is sufficiently small and m p is sufficiently large, 
and 

sup J(si)p) < sup J(sVp). 

s>0 s>0 

Namely sup s>0 J(sv p ) < c p , which is in contradiction of our definition of c p . Note that 
v p is not smooth, but can be approximated by smooth, /c-admissible functions. This 
completes the proof of Lemma 2.1. □ 



4.3. The case k = ^. In this case, tl 
that J(v) — > — oo as v — > 0, and also we dc 
any admissible function v with Vol7W Su = 

J(v) = S n/2 (v) - 

is the corresponding functional and £ n /2 
have the following 



le proof of Lemma 4.3 does not apply, due to 
•n't know if S n / 2 (v) is bounded from below for 
1, where 

1 f -2n--e 

/ v n ~ 2 

P+ 1 J(M, 90 ) 

is given in (2.32). However when k = f , we 



Lemma 4.6. Assume that equation (2.1) is variational. Then J 7 n /2(v) is a constant. 
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Proof. When k = ^, we write the equation (2.1) and the functional J r n / 2 in the form 

Vn/2(X(W)) = e~™, 

r n/2 {w) = [ <r n/2 (X(W)), (4.24) 



'(M,g ) 

To prove that J r n /2 is equal to a constant, we have 

- 1 d 



J(M,g )J0 dt 

L Wt (w) 



'0 J(M,g ) 

where w t = tw, wo = 0, L Wt is the linearized operator of cr n / 2 (A(VF)) at w t . By the 
assumption that equation (2.1) is variational, we have (see §2.4) 



/ L Wt (w) = / wL Wt (l) = 0. 

J(M.an) J(M.on) 



f (M,g ) J(M,g ) 

This completes the proof of Lemma 4.6. □ 
By assumption (1.7), we have 

F n /2(v) = c <Y n/2 (S n ) (4.25) 

for some constant cq depending on go). Lemma 4.6 enables us to prove the following 

Lemma 4.7. For 1 < p < the set of solutions of (2.5) is compact. 

Proof. When 1 < p < ^-jr|, the proof is the same as that of Lemma 4.4. 

When p = we use the same argument of Lemma 4.4. Instead of (4.16), we have 
the equation 

<V2(A(V0) =v™ in R n . (4.26) 
By the Liouville theorem [LL1], v must be the function given in (4.20). Hence we have 

/ cr n/2 (X(V))=Y n/2 (S n ). 

By (4.9), we obtain that 

M^ooFnM > Y n/2 (S n ). 

This is in contradiction with (4.25). □ 

By Lemma 4.7, we can prove the existence of solutions of (2.1) by a degree argument, 
see [CGY2, LL1]. We omit the details here. 

4.4. A Sobolev type inequality. As a consequence of our argument above, we 
have the following Sobolev type inequality. 
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Corollary 4.1. Let 2 < k < ^. T7ien i/iere existe a constant C > snc/i £/ia£ t/ie 

[yoZ(.M 3 )]^r < C[ / <r fc (A(A ff ))dwZ ff ]^ (4.27) 

J M 

4 

/io/ds /or any conformal metric g = v n ~ 2 go with v G 
Proof. Note that (4.27) is equivalent to 

/* 2t7 f 2n — 4k ~\l n — 2 

[ v — ] 2n <C[ v^^a k (\(-))] 2n ~ 4 \ (4.28) 

J(M,g ) J(M,go) v 

which is equivalent to (4.18). □ 

Remark 4.1. The Sobolev type inequality (4.27) is similar to (2.17) of I = 0, k > 1, and 
was proved in [GW2] for locally conformally flat manifolds. If M. is locally conformally 
flat and / = fi(e~ 2kw ), the flow (3.21) has a remarkable property. That is by the 
moving plane argument of Ye [Ye] and the conformal invariance of the equation, one 
obtains the gradient estimate (3.23) at all time t, with the upper bound C depending 
only on the initial function w(-,0), for any monotone increasing fi satisfying (2.13). 
Therefore by the second derivative estimate (3.26), the solution of (3.21) converges to a 
solution of the /c-Yamabe equation (2.1), and accordingly one also obtains the Sobolav 
type inequality (4.27). Theorem 4.2 shows that the Sobolev type inequality also holds 
on general manifolds provided equation (1.1) is variational. 
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5. Proof of Lemma 2.2 



We let v e be the function given by 



n-2 
2 



-(•0=1^^) ■ (5.1) 



where r = \x\, x e R n , and £ > is a small constant. Let T4 be the matrix relative to 
v e , see (2.2). Then we have 

l: (n-2)v~ 2 I. 



Hence v F is /c-admissible on R n and 



where C n ^ = ^ ! ,^_^, . It follows that 



' w — 2 (T fc (A(— )) = C n>fc / 

R™ ^ ^i?" 



So we have 



2ra 

R™ 



(5.2) 



[Jr^- 2 ] 

In particular we have 

y *( gW )= (n(n-2))* [ri(gW)]fc - (5 - 3) 



In the above i> = i> e and the integrations are independent of e. 

To verify (1.7), it would be natural to use the function (5.1) as a test function, as 
in the case k = 1 [Aul, SI]. However on a general manifold, the function v e , where r 
denotes the geodesic distance from a given point, is /c-admissible only when r < Ce 1 ! 2 . 
It seems impossible to find an explicit test function. 

Instead we shall deduce (1.7) directly from (1.6). First note that by assumption, there 
exists a function v > such that g = t; 4 /( n - 2 )g g p fc- Hence ai(\(A g )) > 0. That is the 
scalar curvature of (M.,g) is positive. Hence the comparison principle for the operator 
a 1 (X(A g )) holds on M. 

Let v\ be a solution of the Yamabe problem (with k = 1) such that Qi(fi) < Y"i(5 n ), 
where 

<9i(u) = 



f M vai(X(V)) 



[j- v 2n/(n-2)](n-2)/r 
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Let Vk be the solution of 



/-. rt+2 

*k(\(V)) = C n;kVl n - 2 in M (5.4) 



By Lemma 2.3(vi), we have 

~ Avk + 4{n-l) RVk = ai ^ Vk ^ - n{n ~ 2 ) u i 



n-2 „ . £±§ 



Since i>i satisfies 

n — 2 "+2 

-Av + -Rv = n(n - 2)v^ 2 , 

4(n - 1) 1 

by the comparison principle, 

Vk > V! (5.5) 

Now writing 

[/ Ml ,w<n-3) dl , ,J<"- 2 w"' 

we claim that 

Q fc K) < n(5 n ), (5.6) 
namely (1.7) holds. Indeed, when k > 2, we have — /c^f < 0. Hence by (5.5), 

2n u "+2 2ra )„ ra+2 

„_2 K n -2 > n-2 K ™-2 



Hence 



f In i n+2 /• 2ra / rt+2 , ra+2 

/ ^"" 2 n - 2 Vk(\(V k ))dvol g <C n , k vr 2 n ~ 2 v x n ~ 2 dvol g 

JM J B p 

< C n ^k / V l~ 2 dvolg 



B n 



and 



Therefore we obtain 



C 2n f 2n 

/ w fc "" 2 cfeo/ 3 > / v{~ 2 dvolg. 
Jm Jb p 

Qk(v k ) < C n , k [ [ vf~ 2 dvol g ] 2k/n , (5.7) 
Jm 

so that (5.6) follows from (5.3). 

We remark that a similar argument can be used to prove the inequality Yi(M) < 
Yi(S n ) for some manifolds. 
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6. Some remarks 



6.1. Compactness of the solution set. For the Yamabe problem (k = 1), Schoen 
[S2] has shown furthermore that the set of solutions is compact if the manifold is locally 
conformally flat and not conformally equivalent to the sphere. Schoen's result was ex- 
tended to general compact manifolds for which the positive mass theorem holds, such as 
in low dimensions 3 < n < 7 [LZ, M]. 

When /c > f , the compactness of solutions has also been established in [GV2], for the 
more general equation 

a k (X(A gv )) = fv^, (6.1) 
where / is any positive, smooth function /. Their proof relies crucially on the fact 

4 

that the Ricci curvature Ric 9v > Cg v if g v = go is a solution to (6.1). For the 
convenience of the reader we indicate the idea of their proof here. From the positivity of 
the Ricci curvature, the volume of (M,g v ) is uniformly bounded. Hence if there exists 
a sequence of solutions {v k } with supf^ — > oo, there are at most finitely many blow- 
up points P =: {po,pi,--- ,p s }- By the interior first and second derivative estimates 
(3.23) and (3.26), v k / inf Vk converges locally uniformly on M. — P to a C 1 ' 1 function v 
with a k (\(A gv )) =0. By a technical analysis one has v(x) = C(l + o(l))r 2_n near the 
singularity set P. Since Ric 9v > 0, the ratio h(r) =: Br ^° • is non-increasing, where 
\B r (po)\ denotes the volume of the geodesic ball on (Ai,g v ). On the other hand, since 
v(x) = C(l + o(l))r 2_n , we have lim^oo h(r) = (s + l)w n , where u n is the volume of 
the Euclidean unit ball. Hence s = and (M.,g v ) is isometric to the Euclidean space 
R n , and so M. is conformal to the unit sphere. Note that when 2 < k < ^, the Ricci 
curvature of (M.,g v ) may not be positive anymore. 

6.2. Conditions (CI) and (C2). As indicated earlier, we impose condition (CI) 
so that equation (1.1) is elliptic. If a fully nonlinear partial differential equation is not 
elliptic, little is known about the existence and regularity of solutions. For example it is 
unknown whether there is a local solution to the Monge- Ampere equation detD 2 u = f 
when the right hand side / changes sign, even in dimension two. But possibly condition 
(CI) may be replaced by the positivity of the Yamabe constant Yk{M), as in the case 
k = 2, n = 4 [CGY1, GV1]. 

As for the condition (C2), the variational approach to the k- Yamabe problem is nat- 
ural, as in the case k = 1. Indeed this approach has already been employed in [CGY1, 
CGY2, GW2, GW3], and Theorem 2.1 was proved in [CGY1, CGY2] when n = 4, k = 2, 
and in [GW2, GW3] when M is locally conformally flat. At the moment we are not 
aware of any other possible ways to remove the variational structure condition (C2) for 

the case 2 < k < ^, even in low dimensional cases. 
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6.3. The full &>Yamabe problem [K, La]. We bring to the attention of the readers 
the full /c-Yamabe problem. On a Riemannian manifold (M n , g), one can define a series 
of scalar curvatures 

s k = s k (Riem) = s k (W + A(Dg), (6.2) 

for k = 1, 2, • • • , [||], where Riem, W, A are introduced at the beginning in the introduc- 
tion. The /c-scalar curvature can also be expressed simply as 

Sk = J^y c2kRiemk ' ( 6 - 3 ) 

where c is the standard contraction operator, and Riem k = Riem o • • • o Riem is the 
product introduced in [K]. 

When k = 1, si is the usual scalar curvature. When n is even, s n /2 is the Lipschitz- 
Killing curvature. Furthermore if M. is a hypersurface, the /c-scalar curvature is the 
2k th mean curvature H^k-, which is equal to the 2k th elementary symmetric polynomial 
of the principal curvatures of the hypersurface, which is intrinsic quantity [Sp]. When 
M is locally conformally flat, then the Weyl curvature in (6.2) vanishes, and s k turns 
out to be the fc-curvature given in (1.1). 

The full /c-Yamabe problem concerns the existence of a conformal metric such that the 
fc-scalar curvature is a constant. This problem coincides with the fc-Yamabe problem for 
locally conformally flat manifolds. The corresponding equation of the /c-Yamabe problem 
is always variational, as in the case k = 1 [La]. However the equation is of mixed type 
in general. 
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